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Project  Objective 

Research,  design,  evaluate,  and  implement  self-adaptive  mechanisms  and 
algorithms  to  improve  the  performance  of  service  discovery  protocols 

for  use  in  fault-tolerant  networks. 

Project  Plan  -  Three  Phases 

•  Phase  I  -  characterize  performance  of  selected  service  discovery  protocols 
(Universal  Plug-and-Play  -  UPnP  -  and  Jini)  as  specified  and  implemented 

■  develop  simulation  models  for  each  protocol 

■  establish  performance  benchmarks  based  on  default  or  recommended 
parameter  values  and  on  required  or  most  likely  implementation  of  behaviors 

•  Phase  II  -  design,  simulate,  and  evaluate  self-adaptive  algorithms  to  improve 
performance  of  discovery  protocols  regarding  selected  mechanisms 

■  devise  algorithms  to  adjust  control  parameters  and  behavior  in  each  protocol 

■  simulate  performance  of  each  algorithm  against  benchmark  performance 

■  select  most  promising  algorithms  for  further  development 

•  Phase  III  -  implement  and  validate  the  most  promising  algorithms  in  publicly 
available  reference  software 
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Dynamic  Discovery  Protocols  in  Essence 

Dynamic  discovery  protocols  enable  network  elements  (including  software 
clients  and  services,  as  well  as  devices): 

(1 )  to  discover  each  other  without  prior  arrangement, 

(2)  to  express  opportunities  for  collaboration, 

(3)  to  compose  themselves  into  larger  collections  that  cooperate  to  meet 
an  application  need,  and 

(4)  to  detect  and  adapt  to  changes  in  network  topology. 


Selected  First-Generation  Dynamic  Discovery  Protocols 
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Two  Party  vs.  Three  Party  Architectures 


ird  Party 


SM 


Service 
Manager 


Service 


Rep 


Service 
Descriptio 


Parameter 
Change 
Notification 
Cache 


SCM 


Notification 

Request 


0..*  \|/ 

« repositor  y  e  ntr  y» 
Parameter  Notification  Recjuest 

LOCAL  C  AC H  E  MANAGER 

1 

<<repository» 

Parameter 

Change 

Ag  i  ng  Tas  k( ) 

Service  Cache 

(from  Data  View) 

Notification 

Request 


Local 

Cache 

Manager 


Service 

Cache 


\ 


1/18/2002 


5 


NIST 

National  Institute  of  Standards  and  Technology 

Technology  Administration,  U,S,  Department  of  Commerce 


Technical  Approach  to  Phase  I  -  Use  Rapide  to 
Model  and  Understand  Fault  Behavior  of  Jini  and  UPnP 
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Specification 


-**3.3  DIRECTED  DISCOVERY  CLIENT  INTERFACE  ** 


—  This  is  used  by  all  JINI  entities  in  directed 

—  discovery  mode.  It  is  part  of  the  SCM_Discovery 

—  Module.  Sends  Unicast  messages  to  SCMs  on  list  of 

—  SCMS  to  be  discovered  until  all  SCMS  are  found. 

—  Receives  updates  from  SCM  DB  of  discovered  SCMs  and 

—  removes  SCMs  accordingly 

—  NOTE:  Failure  and  recovery  behavior  are  not 

—  yet  defined  and  need  reviw. 

TYPE  Directed  Discovery  Client 

(SourcelD  :  IP  Address;  InSCMsToDiscover  :  SCMList;  StartOption  :  DDCode; 
InRequestlnterval :  TimeUnit;  InMaxNumTries  :  integer;  InPV  :  ProtocolVersion) 

IS  INTERFACE 

SERVICE  DDC  SEND  DIR  :  DIRECTED_2_STEP_PROTOCOL; 

SERVICE  DISC  MODES  :  dual  SCMDISCOVERYMODES; 

SERVICE  DDSCMUpdate  :  DD  SCM  Update; 

SERVICE  SCMJJpdate  :  SCMJJpdate; 

SERVICE  DBUpdate  :  dual  DBUpdate; 

SERVICE  N ODE  F AILURES  :  NODE_F ALLURES;  -  events  for  failure  and  recovery. 
ACTION 

IN  Send_Requests(), 

BeginDirectedDiscoveryO; 

BEHAVIOR 

action  animation_Iam  (name:  string); 

MySourcelD  :  VAR  IP_Address; 

PV  :  VAR  ProtocolVersion; 
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Two-Party  (UPnP)  Topology  for  Experiment 
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Three-Party  (Jini)  Topologies  for  Experiment 
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Faults  Interfere  with  Discovery  and  Information  Propagation 


Interface  Failure  -  Tx,  Rx,  or  Both 

■  Due  to  nearby  enemy  jamming  or  other  interference 

■  Due  to  multi-path  fading  during  mobility 


•  Path  Loss  -  Pt-Pt  or  Area-Area  and  Full-duplex  or  Half-duplex 

■  Due  to  persistent  congestion 

■  Due  to  physical  link  cuts 

■  Due  to  enemy  jamming  at  routers 


•  Message  Loss  -  under  both  UDP  and  TCP  (>delay) 

■  Due  to  sporadic  or  distant  enemy  jamming  or  other  interference 

■  Due  to  transient  congestion 

■  Due  to  multi-path  fading  during  mobility 


•  Node  Failure  -  Partial  or  Complete  with  variable  persistence  of  information 

■  Due  to  enemy  bombardment  or  cyber  attacks 

■  Due  to  mobility  associated  with  military  operations 
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Discovery  Systems  Divide  Recovery  Responsibilities: 
Lower  Layers ,  Discovery  Protocol,  and  Application 


•  Selective  Reliable  Delivery  by  Lower  Layers 

■  TCP  attempts  retransmissions  (basis  for  Jini-RMI  and  UPnP-HTTP) 

■  UDP  messages  in  UPnP  sent  as  multiple  copies 

•  Periodic  Announcements  by  Discovery  Protocol 

■  Allows  caching  nodes  to  discard  information  when  TTL  expires 

■  Jini  includes  aggressive  search  at  node  start  up,  while  UPnP  permits 
nodes  to  undertake  aggressive  search  at  any  time 

•  Periodic  Refreshing  of  Resources  Required  by  Discovery  Protocol 

■  Allows  resource  owner  to  free  resource  when  refresh  period  expires 

•  Remote  Exceptions  Issued  by  Protocol  Over  TCP  Links 

■  Allows  application  to  take  recovery  action:  Ignore?  Retry?  Discard 
knowledge  of  service  or  resource? 
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Understanding  Information  Propagation  in  Discovery 
Architectures  during  Interface  Failure 

How  do  various  service  discovery  architectures,  topologies,  and  fault-recovery 
mechanisms  perform  under  deadline  during  interface  failure? 


Outline  of  Experiment _ 

•  Deploy  models  of  two-party  (UPnP)  and  three-party  (Jini)  architectures  with 
one  SM  and  five  SUs  (for  Jini  include  two  topologies  -  one  and  two  SCMs). 

•  Ensure  initial  discovery  and  information  propagation  completed. 

•  Introduce  a  change  in  the  service  description  at  the  SM,  and  establish  a 
deadline  for  propagating  the  new  information  to  all  SUs. 

•  Measure  the  number  of  messages  exchanged  and  the  latency  required  to 
propagate  the  information  to  all  SUs,  or  until  the  deadline  arrives,  under  two 
different  propagation  mechanisms:  polling  and  eventing. 

•  Repeat  this  experiment  while  varying  the  percentage  of  interface  failure  time 
for  each  node  up  to  75%  (in  increments  of  5%). 
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Information-Propagation  Mechanisms  for  Experiment 


Two 

Party 


Three 

Party 


Polling 


Eventing 


SM 


Change 

Service 


SU 


Get  Description 


Get  Description 


Get  Description 


SM 


SU 


Change 

Service 


Notification  Request 


◄ — 

Get  Description 

Notification 

◄ — 

Get  Description 

W 

SM 


SCM 


SU 


SM 


SCM 


SU 


Find  Service 


Change  Service 


Change  Service^ 


Notification 


Request 
Find  Service 


Notification 


1/18/2002 


12 


nist 


National  Institute  of  Standards  and  Technology 

Technology  Administration,  U,S,  Department  of  Commerce 


Interface-Failure  Model  for  Experiment 
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Choose  a  time  to  introduce  the  change  [uniform(Q,  D/2)] 

For  each  node,  choose  a  time  to  introduce  an  interface  failure 
[uniform(Q,  D-((D-Q)*F))] 

When  each  interface  failure  occurs,  choose  the  scope  of 
the  failure,  where  each  of  [Rx,  Tx,  Both]  has  an  equal  probability 


Q  =  end  of  quiescent  period  (100  s  in  our  experiment) 

D  =  propagation  deadline  (5400  s  in  our  experiment) 

F  =  Interface  Failure  Rate  (variable  from  0%  -  75%  in  5%  increments  in  our  experiment) 
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Our  Model  Responses  to  Remote  Exceptions  (“Approximately”) 


•  Ignore  REX  Received 

■  When  replying  to  a  remote-method  invocation 

■  When  attempting  to  cancel  a  lease 

■  When  attempting  to  renew  a  lease  (But  then  attempt  to  obtain  a  new  lease) 

•  Retry  Operation  for  Some  Period  of  Time  -  Then  Quit  If  Not  Successful 
(If  Quitting,  Eliminate  Local  Knowledge  of  Discovered  Entity) 

■  When  attempting  to  register  for  notification  events 

■  When  a  UPnP  SU  requests  service  descriptions 

•  Retry  Operation  Persistently  as  Long  as  Application  Executes 

■  When  a  Jini  SM  attempts  to  register  a  service  description  with  a  SCM 
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Metrics  Devised  for  Information-Propagation  Experiments 


•  Update  Responsiveness  (R) 

Assuming  that  information  is  created  at  a  particular  time  and  must  be  propagated 
by  a  deadline,  then  the  difference  between  the  deadline  and  the  creation  time 
represents  available  time  in  which  to  propagate  the  information.  Update 
Responsiveness,  R,  measures  the  proportion  of  the  available  time  remaining  after 
the  information  is  propagated.  [1  =  all  time  remains  and  0  =  no  time  remains] 


•  Update  Effectiveness  ( U ) 

Update  Effectiveness,  U,  measures  the  probability  that  information  will  propagate 
successfully  to  a  SU  before  some  deadline,  D.  [1  =  information  will  be  propagated 
and  0  =  information  will  not  be  propagated] 


•  Update  Efficiency  (E) 

Given  a  specific  topology  of  SUs  and  SMs  in  a  discovery  system,  examination  of 
the  available  architectures  (two-party  and  three-party)  and  mechanisms  (polling 
and  eventing)  will  reveal  a  minimum  number  of  messages  that  need  to  be  sent  to 
propagate  information  from  all  SMs  to  all  SUs  in  the  topology.  Update  Efficiency, 
E,  can  be  measured  as  the  ratio  of  the  minimum  number  of  messages  needed  to 
the  actual  number  of  messages  observed.  [1  =  only  minimum  number  of 
messages  needed  and  0  =  infinite  number  of  messages  needed] 
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Responsiveness:  UPnP  (2-Party)  vs.  Jini  (3-Party) 
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Efficiency:  UPnP  (2-Party)  vs.  Jini  (3-Party) 
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Plan  for  the  Next  Six  Months 

•  Submit  two  papers  on  recent  results:  MILCOM  2002  and  3rd  International  Workshop 
on  Software  Performance 

•  Complete  characterization  of  UPnP  and  Jini  behavior  (ending  Phase  I) 

■  Information  propagation  during  message  loss 

■  State  recovery  during  node  failure 

■  Performance  under  increasing  network  size 

•  Develop  and  document  ideas  for  initial  set  of  self-adaptive  discovery  mechanisms 
(beginning  Phase  II) 

•  Complete  scalable  (up  to  500  nodes)  discrete-event  simulation  model  of  UPnP  - 
based  on  source  code  from  Intel’s  Linux  SDK  for  UPnP  (preparing  our  Phase  II 
models  for  easy  conversion  to  implementation  during  Phase  III) 

•  Extend  our  existing  generic  structural  model  of  service  discovery  systems  to  cover 
behavior,  message  vocabulary,  and  consistency  conditions  (we  see  this  as  a 
community  service) 
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Conclusions 


•  Emerging  industry  discovery  protocols  provide  robustness  properties  through  a 
division  of  responsibilities  among:  lower  layer  protocols,  discovery  protocols,  and 
applications 

•  Characterizing  the  behavior  and  robustness  of  commercial  service  discovery  protocols 
is  a  necessary  pre-condition  to  developing  and  evaluating  adaptation  mechanisms 
intended  to  improve  the  performance  of  such  protocols 

•  We  described  an  approach  to  understand  the  behavior  of  service  discovery  protocols 
in  the  face  of  various  faults:  interface  failure,  message  loss,  and  path  and  node  failure 

(To  learn  more  about  the  approach,  see:  “Analyzing  Properties  and  Behavior  of  Service  Discovery  Protocols  Using 
an  Architecture-based  Approach”,  C.  Dabrowski  and  K.  Mills,  Proceedings  of  Working  Conference  on  Complex  and 
Dynamic  Systems  Architectures,  sponsored  by  DARPA  DASADA  program) 

•  We  applied  the  approach  to  characterize  the  performance  of  two  different  mechanisms 
(polling  and  eventing)  for  information  propagation  in  various  service  discovery 
architectures  (2-party  and  3-party)  and  topologies  (one  and  two  SCMs)  during 
interface  failure 

•  We  are  currently  conducting  characterizations  of  performance  in  the  face  of  message 
loss  and  node  failures 
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Slides  Containing  Additional  Details 
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Equating  a  Generic  Structural  Model  of  Service  Discovery 
Architectures  to  Selected  Commercial  Discovery  Systems 
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The  Six  Combinations  of  Architecture,  Topology,  and 
Consistency-Maintenance  Mechanism  Used  in  Experiments 


Architectural  Variant  Protocol  Basis  Consistency-Maintenance  Mechanism 
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Specific  Division  of  Failure-Recovery  Responsibilities 

Used  in  Experiments 
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Party  Mechanism  Architecture  (UPnP)  Architecture  (Jini) 


Lower-Layer 

Protocols 

UDP 

No  recovery 

No  recovery 

TCP 

Issue  REX  in  30-75  s 

Issue  REX  in  30-75  s 

Discovery 

Protocols 

Lazy 

Discovery 

SM:  announces  every  1800  s 

SCM:  announces  every  120  s 

Aggressive 

Discovery 

SU:  issues  Mseanch  every  120  s 
(after  purging  SD) 

SU  and  SM:  issue  seven  probes 

(at  5  s  intervals)  only 
during  startup 

Application 

Software 

Ignore  REX 

SU:  HTTP  Get  Poll 

SM:  Notification 

SU:  FindService  Poll 

SCM:  Notification 

Retry  in  1 20  s 
after  REX 

SU:  HTTP  Get  after  discovery 
(retry  up  to  three  times) 

Subscribe  requests 

SM:  depositing  or  refreshing  SD 
copy  on  SCM 

SU:  registering  and  refreshing 
notification  requests  with  SCM 

Discard 

Knowledge 

SU:  purge  SD  after  failure  to 
receive  SM  announcement 
within  1800  s 

SU  and  SM:  purge  SCM  after 

540  s  of  continuous 

REX 
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Significant  Parameters  and  Values  Used  in  Experiments 


Parameter 

Value 

Behavior  in  both 
two-  and  three- 
party  architectures 

Polling  interval 

180  s 

Registration  TTL 

1800  s 

Time  to  retry  after 

REX  (if  applicable) 

120  s 

UPnP-specific 
behavior  for  two- 
party  architecture 

Announce  interval 

1800  s 

Msearch  query  interval 

120  s 

SU  purges  SD 

At  TTL  expiration 

J  ini-specific 
behavior  for  three- 
party  architecture 

Probe  interval 

5  s  (7  times) 

Announce  interval 

120  s 

SM  or  SU  purges  SCM 

After  540  s  with  only  REX 

Interface  failure 
parameters 

Failure  incidence 

Once  per  run  for  each  node 

Failure  scope 

Transmitter,  receiver,  or 
both  with  equal  likelihood 

Failure  duration 

5%  increments  of  5400  s 
from  0  to  75% 

Transmission  and 
processing  delays 

UDP  transmission 
delay 

1 0  us  constant 

TCP  transmission  delay 

10-100  us  uniform 

Per-item  processing 
delay 

1 00  us  for  cache  items 

1 0  us  for  other  items 
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Console  Output  from  a  Sample  Experiment  Run 


Rate  -  5 

Run  number  -  21 


SM 

1 

OUT 

Interface 

down 

3  65, 

up 

635 

SCM 

:  l 

OUT 

Interface 

down 

2417, 

up 

2687 

SCM 

:  2 

IN 

&  OUT  Interface 

down 

519, 

up 

789 

SU 

l 

IN 

Interface 

down 

2238, 

up 

2508 

SU 

2 

IN 

Interface 

down 

3256, 

up 

3526 

SU 

3 

IN 

Interface 

down 

207, 

up 

477 

SU 

4 

OUT 

Interface 

down 

2876, 

up 

3146 

SU 

5 

IN 

Interface 

down 

4478, 

up 

4748 

Performance : 


SM 

1 

346.00000 

346.00000 

6 

17 

SCM 

1 

346.00000 

346.00016 

61 

102 

SCM 

2 

346.00000 

346.00015 

61 

105 

SU 

1 

346.00000 

346.00109 

0 

11 

SU 

2 

346.00000 

346.00109 

0 

11 

SU 

3 

346.00000 

5400.00000 

4 

11 

SU 

4 

346.00000 

346.00109 

0 

11 

SU 

5 

346.00000 

346.00114 

0 

11 

1/18/2002 


26 


NIST 

National  Institute  of  Standards  and  Technology 

Technology  Administration,  U,S,  Department  of  Commerce 


Update  Responsiveness  ( R ) 


Let  D  be  a  deadline  by  which  we  wish  to  propagate  information  to  each 
service  user  (SU)  node  ( n )  in  a  service  discovery  topology. 

Let  tc  be  the  creation  time  of  the  information  that  we  wish  to  propagate,  where  tc  <  D. 

Let  tu(n)  be  the  time  that  the  information  is  propagated  to  SU  n,  where  n  =  1  to  N,  and 

N  is  the  total  number  of  SUs  in  a  service  discovery  topology. 

Define  information-propagation  latency  (Z_)  for  an  SU  n  as: 

Ln  =  %(„)  -  fc)/(max(D,  tU(n))  -  tc). 


Define  update  responsiveness  (R)  for  an  SU  n  as: 
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Update  Effectiveness  ( U ) 

Let  the  definitions  related  to  Update  Responsiveness,  R,  hold. 

Let  X  represent  the  number  of  runs  during  which  a  particular  service  discovery 
topology  is  observed  under  identical  conditions. 

Recalling  that  N  is  the  total  number  of  SUs  in  a  service  discovery  topology, 
define  the  number  of  SUs  observed  under  identical  conditions  as: 

O  =  X  N. 

Define  the  probability  of  failure  to  propagate  information  to  an  SU  as: 

P(F)  =  (count(R..  ==  0 ))/0,  where  /  =  1  ..A/ and  j  =  1  ..X. 

'J 

Define  the  Update  Effectiveness  for  a  given  set  of  conditions  as: 

U  =  1  -  P(F). 
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Update  Efficiency  (E) 


Let  the  preceding  definitions  associated  with  Update  Responsiveness  and  Update 

Effectiveness  hold. 

Let  M  be  the  minimum  number  of  messages  needed  to  propagate  information  from 

all  SMs  to  all  SUs. 

Let  S  be  the  observed  number  of  messages  sent  while  attempting  (failures  may  occur) 
to  propagate  information  from  all  SMs  to  all  SUs  in  a  given  run  of  the  topology. 

Define  average  Update  Efficiency  as: 

Eavg  =  ( sum(M/Sk))/X ,  where  k=  1  ..X. 
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Summary  Statistics  for  Performance  of 
Each  Combination  on  Each  Metric 


Mean  (across  all  interface-failure  rates) 

Median 

Responsiveness 

Effectiveness 

Average 

Efficiency 

Two-Party  Notification 

0.663 

0.921 

0.212 

Two-Party  Polling 

0.615 

0.973 

0.251 

Three-Party  Notification 
(Single  SCM) 

0.601 

0.894 

0.389 

Three-Party  Polling 
(Single  SCM) 

0.530 

0.911 

0.201 

Three-Party  Notification 
(Dual  SCM) 

0.655 

0.942 

0.221 

Three-Party  Polling 
(Dual  SCM) 

0.587 

0.927 

0.110 
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95%  C.IJ.  for  Each  Metric-Combination  at  Selected  Failure  Rates 


Responsiveness 

Effectiveness 

Efficiency 

5% 

40% 

75% 

5% 

40%. 

75% 

5% 

40% 

75% 

Two-Party  Notification 

1.000 

1.000 

0.561 

0.783 

0.111 

0.162 

0.970 

0.977 

0.954 

0.966 

0.709 

0.787 

0.354 

0.467 

0.065 

0.220 

0.031 

0.354 

Two-Party  Polling 

0.975 

0.980 

0.501 

0.849 

0.076 

0.138 

1.000 

1.000 

0.993 

0.993 

0.760 

0.826 

0.501 

0.666 

0.031 

0.230 

0.042 

0.059 

Three-Party  Notification 
(Single  SCM) 

1.000 

1.000 

0.605 

1.000 

0.042 

0.095 

0.993 

0.993 

0.939 

0.955 

0.521 

0.652 

0.827 

1.000 

0.099 

0.504 

0.033 

0.320 

Three-Party  Polling 
(Single  SCM) 

0.974 

0.980 

0.244 

0.412 

0.043 

0.083 

1.000 

1.000 

0.946 

0.960 

0.660 

0.753 

0.387 

0.512 

0.043 

0.173 

0.040 

0.164 

Three-Party  Notification 
(Dual  SCM) 

1.000 

1.000 

0.562 

1.000 

0.099 

0.143 

0.970 

0.977 

0.977 

0.983 

0.730 

0.803 

0.335 

0.599 

0.035 

0.290 

0.009 

0.096 

Three-Party  Polling 
(Dual  SCM) 

0.974 

0.986 

0.391 

0.543 

0.056 

0.096 

1.000 

1.000 

0.939 

0.955 

0.660 

0.753 

0.218 

0.273 

0.033 

0.103 

0.019 

0.059 
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